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REMARKS 

After entry of the foregoing amendment, claims 26-70 are pending in the appHcation. 

The Section 112 issue has been addressed by removing the term "computations" in claim 
47. A similar change has been made in claims 62 and 70. 

Conceming audio signals, essentially all of the references to audio in the priority 
specifications concern steganographically encoding a signal in such audio. 

The Field of the Invention paragraph in the 1993 priority specification noted: 

The present invention relates to the field of identifying signals and images as to 
origin and ownership. Specifically, it relates to the need in the commercial audio and 
visual industry to unequivocally identify a multi-generation copy of source material 
based on objective methods rather than purely subjectively and perceptually based 
opinion. 

The techniques disclosed in this specification for "unequivocally identifying a multi-generational 
copy of source material based on objective methods" rest on steganographic encoding. 
That specification goes on to note: 

What is needed is a much simpler and truly cost effective method for performing a 
positive identification between a copy of an original signal and the original. This method 
should not only be able to perform positive identification, it should also be able to relate 
version identification of sold copies in order to better pinpoint the point of sale. The 
method should not compromise the innate quality of material which is being sold, as does 
the placement of localized logos on images. The method should be robust so that an 
identification can be made even after multiple copies have been made and/or 
compression and decompression of the signal has taken place. The identification method 
should be largely uneraseable or %ncrackable. " The method should be capable of 
working even on fractional pieces of the original signal, such as a 10 second **riff' of an 
audio signal or the ''clipped and pasted*' sub-section of an original image. 

The existence of such a method would have profound consequences on audio and 
image piracy in that it could A) cost effectively monitor for unauthorized uses of material 
and perform "quick checks"; B) become a deterrent to unauthorized uses when the 
method is known to be in use and the consequences well publicized; and C) provide 
unequivocal proof of identity, similar to fingerprint identification, in litigation, with 
potentially more reliability than that of fingerprinting. 
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In accordance with a preferred embodiment of the invention, a computer systems 
is provided with associated means for manipulating either digital audio signals or digital 
images. In cases where original material is in "non-digital" form, such as on audio tape 
or on a photograph, means for creating a high fidelity digital copy of the material is 
included in the preferred embodiment This physical system will be referred to as the 
"Eye-D" workstation or system which serves as a concise trade name. The Eye-D system 
embeds an imperceptible global signal either directly onto the digital original or onto 
the "digitized copy" of the original if it was in a non-digital form to begin with. The new 
copy with the embedded signal becomes the material which is sold while the original is 
secured in a safe place. The new copy will be nearly identical to the original except 
under the finest of scrutiny; thus, its commercial value will not be compromised. After 
the new copy has been sold and distributed and potentially distorted by multiple copies, 
the present disclosure details a method for positively identifying any suspect signal 
against the original. 

The embedding of an imperceptible signal is steganographic encoding of a signal.' 
A variety of other excerpts might be cited, but the foregoing are believed to establish the 
requisite support. 

Claim 26 stands rejected over Moses (5,404,377) in view of Schwab (5,134,496). Claim 
50 stands rejected over O'Grady (4,969,041) in view of Gniewek (5,265,082). AppHcant 
respectfully traverses these rejections. For example, the Action wrongly characterizes Schwab as 
teaching use of imperceptibly embedded signals to prevent copying. However, Schwab is 
understood to teach an arrangement in which "code sequences are visually manifested as 
dropouts on a video monitor" (Abstract) and as a "video imperfection" (col. 2, line 26). This is 
not "imperceptible" (even if, to some viewers, it may be "acceptable"). No reference to 
imperceptibility is found in Schwab. 


* Steganography is the art/science of hiding one infonnation signal in another carrier. Two references are 

enclosed to illustrate this meaning. 

One is Kawaguchi, "Principles and Applications of BPCS-Steganography," Proc. Of SPIE, Vol. 3528, pp. 
464-73, Nov 2-4, 1998, which states at page 1 "Steganography is a technique to hide secret information in some 
other data (we call it a vessel) without leaving any apparent evidence of data alteration." 

The other is WO 02/13436, which states at page 1, ""Data hiding or steganography is the art of hiding a 
message signal in a host signal, without any perceptual distortion of the host signal. . . Though steganography is 
often confused with the relatively well-known cryptography, the two are but loosely related. Cryptography is about 
hiding the contents of a message. Steganography, on the other hand, is about concealing the very fact that a message 
is hidden. Steganography may be considered as communication through subliminal channels, or secret 
communications." 
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Likewise, the analysis of claim 50 is believed to be flawed. The Examiner tries to equate 
the claimed "plural sequential series of audio data" to plural frames of video in O'Grady. 
However, applicant's claim requires processing " a plurality of series of said encoded audio 
signal" with the processing unit to extract the auxiliary data. O'Grady, in contrast, doesn't work 
this way. His elements 68/70 are understood to operate on less than a single frame (i.e., the data 
period T referenced at column 3, line 43 is said to be 7050 samples - less than a frame, as 
indicated at column 5, line 37). Thus, the parallel perceived by the Examiner (i.e., decoding 
extending across plural frames) is not understood to be present. 

Since the Action has not established a prima facie case under § 103, ftirther comments on 
the rejections (e.g., the alleged teachings of Gniewek, and the proposed combination) are not 


No rejection was made as to claims 51-64; these are understood to have been objected-to 
for the dependence on rejected claim 50, but should now likewise be in condition for allowance. 

In view of the foregoing, the application is believed to be in condition for allowance, and 
action to that end is solicited. 


belabored. 


Respectfully submitted. 


Date: June 8, 2004 
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Principle and applications of BPCS-Steganography 

Eiji Kawaguchi* and Richard O. Eason** 

♦ Kyushu Institute of Technology, Kitakyushu, Japan 
** University of Maine, Orono, Maine 04469-5708 

ABSTRACT 

Steganography is a technique to hide secret information in some other data (we call it a vessel) without leaving any apparent 
evidence of data alteration All of the traditional steganographic techniques have limited information-hiding capacity TTiey 

^.1?" ' i f ''^ '""""•^ °f ™^ ^''"^ P"'«='P'« °f '"hniqL L either to 

rcptew a special part of the frequency components of the vessel image, or to replace all the least significant bits of a multi- 
valued image with the secret informauon. 

Our new steganography uses an image as the vessel data, and we embed secret information in the bit-planes of the vessel 
THIS technique makes use of the characteristics of the human vision system whereby a human cannot perceive any shape 
iTl^r 211.' binary pattern. We can replace all of the "noise-like" regions in the bit-planes of the ve^ 

wpJriSt^uTajSSbws' '° investigate this technique in depth. The merits of BPCS-Steganography found by ih; 

1 . The information hiding capacity of a true color image is around 50%. 

2. A sharpening operation on the duimny image increases tiie embedding capacity quite a bit 

3. Canonical Gray coded bit planes are more suitable for BPCS-Steganography tiian the standard binary bit planes 

4. Randomization of the secret data by a compression operation makes the embedded data more intangible 

bSdS^T °^ ''.^''^^■^•*8^°8raphy program for each user is easy. It fiirthcr protects against eavesdropping on the 


Keywords: steganography. data hiding, information hiding. BPCS. digital picture envelope, vessel image, dummy image, 
encryption, compression, bit plane / & • 


1. INTRODUCTION 


Internet communication has become an integral part of the infrastructure of today's world. The information communicated 
comes in numerous forms and is used in many applications. In a large number of Uiese applications, it is desired Uiat the 
communicauon be done in secrete. Such secret conmiunication ranges from tiie obvious cases of bank transfers, corporate 
communications, and credit card purchases, on down to a large percenuge of everyday email. With email, many people 
wrongly assume tiiat their coimnunication is safe because it is just a small piece of an enormous amoum of data being se.-tt 
worldwide. After all. who is going to sec it? But in reality, the Internet is not a secure medium, and there are programs "out 
there which just sit and watch messages go by for interesting information. 

Encryption provides an obvious approach to information security, and encryption programs are readily available. However. 
!r.!!^P B° " " containing "interesting" information, and die encrypted message becomes subject lo 

attack. Furthermore, in many cases it is desirable to send infonnation wiUiout anyone even noticing tiiat information has been 

M P;*^.f ^PV^^^ to information security. In steganography. dau is hidden inside a vessel or 

container that looks like it contains only someUiiog else. A variety of vessels arc possible, such as digital images, sound dips, 
and even executable files. In reccm years, several steganographic programs have been posted on Internet home pages. Most 
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di^t^w^. 1"*°!^")? '^'^ °^ information. Some of them use the least significant bits of the ima« 

some cases the cLnical'GST^g^ylm'^^^^^^ "t^^"^' ^^"^-^ ^~«"« ^y-" (^BQ.L in 

2. THE COMPLEXITY OF BINARY IMAGES 

The delinitioii of image complexity 
i^^gcl^ompt":^^^^^^^ 

number of color-changes O^elL ^. ^^c.l^J^ '^ black-and-wh.te border eqiuds to the summation of the 

background pixeSS SHeJ^JfoM ' — by white 

We will define the image complexity a by the following. 


a = K 

(I) 


The max. possible B - W changes in the image 
Where, k is the toul length of black-and-white border in the image. So. the value ranges over 

O^a.l. 

3. ANALYSIS OF INFORMATIVE AND NOISE-LIKE REGIONS 

!"n^gr2;V^ral^S^;a^ 'l^tJ^^^^^^^^ •^•^ - --ere such binary 

P-ems are noise-like. We wi7igii;?r„Sira ^^iMa^n^^roVoTfbi:^^^^ 
1- Cotxiugation of a binary image 
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W B Wc Be 

Fig. 1 Illuscration of each binary pattern (N=4) 

1. The foreground area shape is the same as P. 

2. The foreground area has the Be paltcm. 

3. The background area has the Wc pattern. 

sssrcir ;':.s,^:^°j:~o^ ^::.^r '-'-^ ^ - '« - 

A) P* = P®Wc 

B) (?♦)♦ = p (3) 

C) P»*p (4) 
The most important property about conjugation is the following <5> 

D) Ut a (P) be the complexity of a given image P. then we have. 


a(P*)= l-a(P). 


(6) 


It is^evident that the combination of each local conjugaUon (e.g.. 8 x 8 area) makes an overall conjugation (e.g.. 512 x 512 
2. Criterion to segment a blt-plane into Informative and nolse-llke regions 

Zl7cS!^^ T "'""^ '^""^ P^"*- a. noise-like with regard to the 

rviii"^™:r„s:c.^u:^^^^^^ 

any means. ' ^ P**"*™- However. 2« is too huge to make an exhaustive check by 

of a. This simulates ti,e dis Judon o> 2" Wn^ paS^ ' generated patterns in tem« 

average value of ti,e complex^ra wJelt Sr^riT^H '^"'^ U.eo«.m'The 
("Sigma- in Fig. 2) ^ "^"^ O-^^ in a. We denote this deviation by a 

^rer-ll^hTcriS^^^^^^^^ rj3LX«e.^ ''^"^ 
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H£tx9gtam of 
Randan 89^ 

PatlBm OmpVnrfy 

Nozm alD Jst2±utzsi 
Curve 


-5-4-3-2-1 0 1 2 3 4 .5 
CcrapJfesly h S^a 
Fig. 2 Histogram of randomly generated 8x8 binary patterns 

L°vfJ:l^eTvlcSTv^^^ If we replace a., the .oca, areas 

is no longerWftht^ L^1,";fi^ P^^^ we can discard more. If U,e quality 

as the threshold vklue. ''^^ W o = Ot is the minimum complexity value to be good, such ot is used 

J*" SvelSS''(;Z!^;?T''"" " « ^^J" "Picture-Hke" after 

S"?n a'bSS? = ^ <=<""P'«i'y th^^hold to divide informative and noise-like 

We made a "random pattern replacing" experiment on a bit-plane of a color image. Fig. 3 iUustmtes the result 




A) Original image B) RandomiraUon (simple side) Q Randomization (complex side) 
Fig. 3 Randomization of the less and the more complex tiian a 0.5 - 8o. 

information. nmdomize the more complex regions Uian OJ - 8a wiOiout losing much of the image 

?4^nTen?ugri: rs^^S-tsS^^^ ^"••^r;? ° - - --"'-''^ 

•« mostiy noise-like bimuy pattons * °' * ' '"'"^ P«««"»- Amazingly. Uie rest (i.e.. 99.9999999999999333%) 

^^^n;;^:^:^ i^gc into 

nt reason we categoric the excessively complicated patterns as "informative" is based on our experiments (7). 
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4. BPCSSTEGANOGRAPHY 

Bit-Plane Complexiiy Segmentation Steganography is our new steganographic technique, which has a large information 
hiding capacity. As was shown in the previous section, the replacement of the complex regions in each bit-plane of a color 
image with random binary patterns is invisible to the human eye. We can use this property for our information hidine 
(embedding) strategy. Our practical method is as follows. 

In our method we call a carrier image a "vessel" or "dummy" image. It is a color image in BMP file format, which hides (or 
embeds) the secret information (files in any format). We segment each secret file to be embedded into a series of blocks' 
having 8 bytes of data each. These blocks are regarded as 8 x 8 image patterns. We call such blocks the secret blocks We 
embed these secret blocks into the vessel image using the following steps. 

1. Transform the dummy image from PBC to CGC system. 

2. Segment each bit-plane of the dummy image into informative and noise-like regions by using a threshold value (tu) A 
typical value is 00=0.3. 

3. Group the bytes of the secret file into a series of secret blocks. - 

4. If a block (S) is less complex than the threshold (oo). Ihen conjugate it to make it a more complex block (S») Tlie 
conjugated block must be more complex than Oq as shown by equation (6). 

5. anbed each secret block into the noise-like regions of the bit-planes (or. replace all the noise-like regions with a series 
ot secret blocks). If the block is conjugated, then record this fact in a "conjugation map." 

6. Also embed the conjugation map as was done with the secret blocks. 

7. Convert the embedded dummy image fi-om COC back to PBC. 

The Decoding algorithm (i.e.. the extracting operation of the secret information from an embedded dummy image) is iust the 
reverse procedure of the embedding steps. / 6 / uic 

The novelty in BPCS-Steganography is itemized in the following. 

A) Segmentation of each bit-plane of a color image into "Informative" and "Noise-like" regions. 

B) Introduction of the B-W boarder based complexity measure (a) for region segmentotion 

C) Introduction of the conjugation operation to convert simple secret blocks to complex blocks 

D) Using CGC image plane instead of PBC plane 


5. EXPERIMENTS 

1. Embedding Capacity 

We have developed BPCS-Steganography programs for both Windows and Unix. In each program, we took an 8 x 8 square 
as the local image size. Fig. 4 (A) is an example of the original dummy image (640x588. full color). (B) is the same imaee 
with all the infonnation of Fig. 5 embedded in it. As indicated in Fig. 5 this embedded information includes a picture of 
Ijncoln the text from four historical documents, and the entire script from seven of Shakespeare's plays. Note that the size of 
the embedded informaUon before compression is almost as great as the image size itself. Furthermore, the embedding 
operauon does not increase the size of the image by even a single byte. Yet. even when viewed on the computer monitor, the 
images before and after embedding are almost indistinguishable from one anodier. 

It should also be noted that the image of Rg. 4 contains a number of large regions that are relatively flat in color. Our BPCS 
technique made litUe use of such regions for embedding, as doing so would introduce noticeable noise in these regions Fig. 
5 presents an example of embedding in a scene with few fiat regions. In this case the image is 617x504 pixeU. (A) shows the 
onginal image, and (B) shows the image after embedding all the information of Fig. 5 plus an additional Shakespearean play. 
.t^I!i^^^^L°l "''^^ ^^"^ compression and 64.184 bytes after. Tlwrefore the total informaui 
embedded m this 93M08 byte image is actually 1.212.744 bytes before compression; i.e., the embedded information cx«ed$ 
the vessel size by 30*! The compressed data size is 505,502 bytes, which is 54% of the vessel size. Even with this much 
intormation embedded in the image, die embedded and original images look nearly identical when viewed on the monitor 
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(A) Original vessel image 

Fig.4 Example of a 


(B) Embedded vessel image 
vessel image 


File 


Lincoln Picture at right (\ pft} 


The Gettysburg Address 


The Declarat ion of Indep endenr#> 


The Constitu tion (with amendments) 


The Maj^na Carta 
Romeo and Juliet 


Hamlet 
Macbeth 


A Midsummer Nighfs Dream 


The Tamin f ^ of the Shrew 


The Tempest 


A Comedy of Errors 
ToUd 


Original 
Size 


66.190 


1^02 


9,553 


56,989 


31.285 


149.097 


188.626 


109.281 


99.623 


128,385 


102.788 


89.525 


1,032^ 


Compressed 
Size 


(A) Summary of embedded data 


14,803 


12,089 


58,829 


74.690 


43.698 


40.242 


42.044 


Fig.5 Files embedded in Fig 4(B) 



(B) Embedded picture 




(A) Original vessel image (B) Embedded vessel image 

Fig.6 Example of a vessel image with fewer flat regions 


Through our embedding experiments, we found that most color images taken by a digital camera can be used as vessel 
images. In almost all cases, the information hiding capacity was nearly 50% of the size of each vessel image. This capacity is 
4 to 5 times as large as currently known steganographic techniques. 

For a given image, embedding capacity can be traded with image quality by altering the complexity threshold. The image of 
Fig. 4 used a threshold of 24 border pixels per 8 x 8 region; therefore regions having more border pixels than this were 
eligible for embedding. Fig. 7 shows how the capacity changes with threshold for this image. For this image a threshold of 
24 seemed optimal, while lower thresholds iniroduced some "noise" to the image. 


Threshold 

Capacity 

Percent 
Of original 

20 

499432 

44% 

25 

469480 

41% 

30 

437240 

38% 

35 

400848 

35% 

40 

361800 

32% 

45 

315432 

27% 


Fig. 7 Capacity vs. Complexity Threshold for the image of Fig. 4 


2. Using Gray Coded Bit-Planes for Complexity Segmentation 

Fig. 8 illusu-atcs the advantage to using Gray Coded bit planes for complexity segmentation. Parts A through C of this figure 
show the PBC red bit planes numbered 3 through 5 for the image of Fig. 5a, while parts D through F show the CGC version 
of these same planes. From looking at such bit planes, one can get a pretty good idea of which regions of the bit plane are 
complex enough to be replaced with informaUon during BPCS embedding. Recall that the goal with BPCS Steganography is 
to use as much of the image as possible for hiding information without appreciably altering the visual appearance of the 
image. 

In comparing these two sets of bit planes, it is evident that the PBC bit planes provide a much greater region for embedding. 
However, subsUntial portions of the regions on the higher bit planes deemed embeddable using PBC are actually relatively 
flat in color. For example, note the wall in the background. This is because of the "Hamming Cliffs" which occur with PBC 
wherein a small change in color affects many bits of the color value. If embedding were to replace the bits in such complex- 
looking but actually relative flat regions, then substantial color changes would occur. As a simple example, consider a region 
where the blue value hovers nearly randomly between the binary values of Oil 1 1 1 1 1 and lOOOOOOO. In this region, every bit 
plane would look complex and would thus appear to be embeddable, while in practice, it would be prudent to only embed in 
the lower one or two planes. Although occurrences such as this where all bits change in a relatively flat region are rare, the 
frequency of occurrence doubles on each lower bit plane. 

CGC images do not suffer from such Hamming Cliffs. Regions which are relatively flat exhibit fewer changes on the higher 
bit planes. Although this limits the amount of space available for embedding, it does so in regions that should not be altered 
in the first place. With CGC, embedding in each region is done on the higher bit planes only to the extent allowed by the 
complexity produced by actual color variation. 
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(A) PBC red plane 3 


(B)PBCrcdplane4 


(C) PBG red plane 5 




(D)CGCredpIane3 (E) CGC red plane 4 (F) CGC red plane 5 

Fig. 8 Comparison of PBC and CGC bit planes 

6. CUSTOMIZATION OF THE PROGRAM 

^J^^-Sr.^.nofs^^^y algorithm has several embedding parameters for a practical program impIementaUon, Some of 

(1 ) The embedding locaUon of the hcader(s) of the secret file(s) 

(2) The embedding direshold, etc 

(7) The compression parameters of the secret file(s) 

that embeds data in an image in a way that is ^SSie byoA« ^ ^ ^ '^'^ °^ P™8n«n 


471 


7. APPLICATIONS 


c«rt radii, b. replace wiu, o«»r (I.TbSl3,^w ftL lt . ""T- "» <"'• " ■i«<l » *« Ini.se ..d 

c»„„ I, „d make I, ,.,dabl. byZlS^ZZ^^ ' *° *•> 

c«om,«io«p™,te.An.mipteofZhTdSrnSX™ ™ > sl«,dmli»l ,o , common m of 


8. CONCLUSIONS AND FUTURE WORK 

in. i. V.,, complex. J. b,„ d,sc.^ri*r,o";r;;s s .7.';xr • 

(.. ^ZC""" '«»«»"■»•.•« »fonn.a,o .nd n.l».,lk. ^ b, .b. oo„pM„ 

O) Hiiuns seo InfonraUv. infomaA>„ only m a ™,y staple blnan, pation 

r^u^^s.T.srr.^issiS^?"""^ "--^ '""^^ i".<. 

developing new applications. * ' '"^""^'"S formalizing the customization parameters, and 
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remove the embedded bits), and a given distortion of the host signal due to information embedding. 
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METHOD AND SYSTEM FOR STEGANOGRAPfflCALLY EMBEDDING 
INFORMATION BITS IN SOURCE SIGNALS 

This invention relates to a method and system for embedding information 
5, bits in a host signal. The embedding may be performed to determine origin of any 
perfect or imperfect copies of the composite (host plus message) signal, or to use 
the host signal as a cover for secret or covert communications, over a charmel 
which is primarily meant for transmitting the host signal only. 

10 BACKGROUND OF THE INVENTION 

Data hiding or steganography is the art of hiding a message signal in a host 
signal, without any perceptual distortion of the host signal. The composite (host 
plus message) signal is also referred to as stego-signal. Though steganography is 
often confused with the relatively well-known cryptography, the two are but 
15 loosely related. Cryptography is about hiding the contents of a message. 
Steganography, on the other hand, is about concealing the very fact that a message 
is hidden. Steganography may be considered as communication through 
subliminal chaimels, or secret communication. 

20 Rapid increases in bandwidth available for dissemination and storage of 

digital content and availability of software tools for editing multimedia content, 
such as, video, images or audio calls for systems and methods to establish origin of 
such content. In addition, large volumes of multimedia content being exchanged 
over insecure channels, such as, the Intemet, provide within themselves secure and 

25 subliminal steganographic channels for secure or secret communications. 


The proliferation of digital multimedia as opposed to conventional analog 
forms, is primarily a resuh of (1) the ease with which digital data can be exchanged 
over die Internet, and (2) the emergence of efficient multimedia data compression 
30 techniques. 

The first reason listed above is also a major cause for concern. Unlimited 
perfect copies of the original content can be made, and distributed easily. It was 
this concem of protecting intellectual property rights of multhnedia data in digital 
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fonn, that primarily triggered researchers to find ways to watermark multimedia 
data. Watermarking the content is done by embedding some data in the host signal 
(original content). The embedded data may be an imperceptible signature, which 
the owner of the multimedia content should be able to extract when a dispute 

5 regarding ownership occurs. 

Data hiding in multimedia could help in providing proof of origin and 
distribution of content. Multimedia content providers would be able to 
communicate with the compliant multimedia players (or renderers) through the 
10 subliminal, steganographic channel. This communication may control or restrict 
access of multimedia content, and carry out e^commerce for pay-per-use 
implementations. 

A typical application of data hiding for multimedia content dehvery may 
15 involve the content providers supplying the raw multimedia data (say a full length 
movie) along with some hidden agents or control data . The job of the distributors 
would be to package the content in some suitable format (such as, MPEG) 
understandable by the player, for distribution of the multimedia through 
DVDs/CDs or live digital broadcasts, or by hosting web sites for downloads or 
20 streaming. The compliant multimedia players, will typically be connected to the 
Internet. 

In conventional multunedia distribution, the content provider looses all 
control over how the multimedia is used/abused the moment it is acquired by 
25 another party. The key idea behind data hiding is to re-establish control whenever 
the content is used. The content provider, by hiding an agent in his raw data, hopes 
to control access to his/her multimedia content. This can be done with the co- 
operation of the players, and an estabUshed protocol for communication between 
the content providers and the compliant multimedia players. 

30 

Data hiding can be broadly classified into two categories depending on 
whether the original content is needed for extraction of the hidden bits: (1) non- 
oblivious methods need the original content for extracting the hidden bits; and (2) 

2 
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on the other hand, oblivious detection methods extract the hidden bits without any 
knowledge of the original. 


In most data hiding methods, sequence of bits to be embedded, viz. B, is 
5 converted to a form suitable for embedding in a cover content. Initially, the bit 
sequence is converted to a signature sequence. Thereafter, the signature sequence 
is embedded in the cover content by an embedding function to obtain the stego- 
content. 

10 From a signal processing perspective; data hiding methods can be classified 

into two categories, depending on the type of embedding and detecting operators. 
The first category includes methods where the embedding fimction adds the 
signature sequence linearly to stego-content, and the detector detects fi:'om the 
stego-content via correlative processing (these methods are referred to as Type I 

15 methods in data hiding literature). In the second category the embedding function 
and the detector are non-linear, typically employing quantizers (these methods are 
referred to as Type II methods in data hiding literature). One of the important 
characteristics of the non-linear methods is their ability to suppress the noise due to 
the original content (or self-noise), even though the original content is not available 

20 at the receiver. 

The present invention provides a unique data hiding technique that 
substantially reduces the effect that noise, distortion or corruption of the host signal 
have on the detected signal so as to greatly enhance the integrity of steganography 
25 techniques employing oblivious detection of the hidden data. The crux of the 
invention is a class of methods referred to as Type HI methods of which Types I 
and II are just special cases. An optimal choice of parameters for the proposed 
Type III methods depending on the engineering constraints, can substantially 
improve the performance of data hiding. 

30 

The present invention also provides many additional advantages, which 
shall become apparent as described below. 
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SUMMARY OF THE INVENTION 

A method for embedding a message signal in a host signal, the method 
comprising the steps of: 
5 (a) embedding the message signal into the host signal, thereby producing 

a stego signal; and 

(b) detecting an estimate of the message signal from the stego signal; 
provided that the detecting step (b) is not an exact inverse of the 
embedding step (a), and the host signal cannot be exactly extracted 
1 0 from the stego signal. 

The embedding step (a) produces a value 6,. in the stego signal from a 
value in the host signal, and wherein the embedding step (a) comprises limiting 

to a limit value ~ , a magnitude of difference between and a, . 

15 

Furthermore, the embedding step (a) employs a continuous periodic 
fimction to produce the stego signal, and wherein the detecting step (b) employs a 
continuoiis periodic ftinction to produce the estimated message signal. The 
continuous periodic ftmction is a triangular ftmction f(x) having a period A, 
20 wherein: 

-^^/(x)<^forallx; 
/(0) = -|; and 

Optionally, the embedding step (a) produces a value in the stego signal 
25 from a value a; in the host signal and a value in the message signal, such that 

the embedding step (a): 

(i) is subject to a maximum distortion constraint P, 

(ii) employs a continuous periodic ftmction having a period A , and 
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(iii) is represented by the function h^^ E{a^,Si), and employs an 
algorithm as follows: 

if rem(—) > — , then P: =3 rem(—) , 

A 2 '4 A 

else Pi =rem(^)'^i 
A 4 

if (I e,. 1> ^) , then = signie,)^ ; 
A 

if 9, > I", then =-e,-; 

if (3^ > 0 , then bf =ai+ , 
10 else = a^. -e^ . 


The method of the present invention is particularly useful when the stego 
signal is corrupted or distorted prior to detecting step (b). In this embodiment 
where the stego signal is corrupted a value in the stego signal is modified after 
15 the embedding step (a) to yield a value in the corrupted or distorted stego 
signal, such that the detecting step (b): 


(i) produces a value s^^ in the estimated message signal from a value 
in a distorted stego signal, 
20 (ii) employs a continuous periodic function having a period A , and 

(iii) is represented by the function ^„ =Z)(c^), and employs an 
algorithm as follows: 

q, =rem(rj-); 
A 

if 9, then ^„-=3j-^,, 
A 

25 else s^^ = . 
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Preferably the host signal is a sequence a^., for / = 1 to N;ihe message 
signal is a sequence s^y for / = 1 to N ; the stego signal is a sequence 6^, for 
/ = 1 to iV'; the corrupted or distorted stego signal is a sequence c^, for 
5 / = 1 to N; and the estimated message signal is a sequence s^f, for 
j=l to 

The embedding step (a) preferably (i) imposes a limit y on a magnitude 

difference between a value in the stego signal that is produced from a value 
10 in the host signal; and (ii) employs a continuous periodic fiinction having a period 

A to produce the stego signal, wherein such the limit ^ period A are 

chosen to minimize a mean square distance between the message signal and the 
estimated message signal, subject to a maximum distortion constraint P of the 
embedding step (a). 

15 

The present invention also provides a method for mapping K information 
bits to a message signal 5,- , / = 1 to N . This method comprising the step of: 

grouping the K information bits together to represent one of 2^ symbols, wherein 
each of the 2^ symbol is mapped to a basis vector or its negative of a 2^"* x2^"' 
20 orthogonal transform matrix. The orthogonal transform matrix is obtained from a 
cyclic* all-pass filter and its circular shifts. The cyclic all-pass filter is preferably 
obtained from a key. 

25 BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of the data embedding, channel and detection 
operation according to the present invention; 


30 


Fig. 2 is a graph depicting a periodic triangular function employed by the 
detector D of Fig. 1; 
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Fig. 3(a) is a graph demonstrating that the distortion introduced during the 
embedding the 5 in ^ (to obtain 5) of Fig. 1 in accordance with Type II will be 

uniformly distributed between - — and + ~ ; 

2 2 

5 

Fig. 3(b) is a graph depicting the distribution of the distortion introduced in 
accordance with the method of the present invention; and 

Fig. 3(c) is a graph depicting the distribution of the limiting noise . 

10 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

The present invention is a method for efficient secure communication over 
subliminal channels provided by multimedia host signals like audio, ^images and 
15 video transmitted over any channel. For example, the host signal may be 
transmitted over the Internet or distributed in storage mediums by other means or 
even transmitted over analog channels, such as, that used for analog television or 
radio broadcasts. Typically the host signal is expected to undergo some distortion 
before it reaches one or many end points where it may be stored or rendered. 

20 

In the method described herein, the host signal may be any form of 
naturally occurring signals, such as, audio, image or video or artificially 
synthesized versions of them. The host signal may further be represented in some 
transform domain. The choice of the transform may depend on the nature of the 

25 application. For example, if the host signal is an image and is* not expected to be 
re-scaled, resized or rotated, any unitary transform may be used. On the other hand, 
if the image is likely to undergo rotation, scaling and/or translation, a Rotation- 
Scale-Translation invariant transform may be used. If the image is cropped, data 
embedding may be performed in many blocks of the image, so that the hidden bits 

30 can be extracted even if one such block survives. In general, the host signal can be 
coefficients of a one-to-one transform or a many-to-one transform. 
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In the method described herein, the host signal can therefore be considered 
as a vector or a sequence of N real or complex numbers, represented by 

5 A sequence of K bits, represented by 

/ = h - jjf], /, 7 = 1/0 fori <y<ii: 
is mapped by a mapping Af to a signature sequence S , 
M :! -^S , where 
5 = [j, ^2 ••• s^] 

10 

An embedder£ embeds the sequence in^ to obtain the stego sequence B, 
5 = £(^,5), where 

the embedding being performed element-wise, 

b2=E(a2,S2) 
15 b,=Eia„s,) 

b^=Eia^,s^) 

subject to the constraint that cl(A,B) < P where d(A,B) is some distance measure 
of signals A and B , and P is the maximum permitted distortion of the host 
signal. In the preferred embodiment the distance measure is the mean square error: 

(_a,-b,f +{a2-b2f +-^i.a,-b,f 


20 


diA,B)^- ^ 


The stego sequence B may undergo some distortion before it reaches the 
detector as C , given by C = 5 + 2 , where 

Z = [z, ••• z^], 
is the noise in the channel used for transmitting the host signal, and 
25 C = [c, ••• c^]. 
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The detector D obtains an estimate of the signature sequence S 

embedded: 

5 The block diagram of data embedding, the channel and detection operation 

is shown in FIGURE 1. 

Based on this generalization of the embedding and detecting functions E 
and D , prior art in this field can be categorized into two types. 

10 

Characteristics of T)q)e I 

• B = E{A,S)^B = A + S 

• D{B):=A'¥S^S 

The above two equations imply that E and D are not inverses. 
15 La addition, if S is known one can obtain the original host sequence 
A from B as A = B-S. 

Characteristics of Type II 

• B = EiA,S) 
20 • D(B) = S 

Unlike Type I methods, the above two equations show that for Type II 
methods E and D are exact inverses. Additionally, unlike Type I methods, it is 
not possible to obtain A exactly, given B and S . 

25 

In the core of this invention is a class of embedding and detection operators 
E and D , we shall refer to as Type III. 

Characteristics of Type III 
30 • B = E{A,S) 

• D{B) ^ S 
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The above two equations illustrate that E and D are not exact inverses (like Type 
I and unlike Type II). Further, given S and B it is not possible to obtain A (like 
Type II and unlike Type I). 

5 In a preferred embodiment described herein, the detector D , where 

is implemented by the following algorithm: 

q. = rami—) ; 
A 

if > - then ^^--qi. 

10 else 5,^ 

In the above algorithm, rem (x) stands for the remainder of a division operation 
(x). For example, 

rem(5/4)=l, rem(2/2)=0, rem(-6/4)-rem(6/4)=2. 

15 The choice of the parameter A is dictated by the distortion constraint P and the 
energy of the channel noise Z . The detector may also be thought of as employing 
a periodic triangular function shown in FIGURE 2, 

3; = f(x) = f(x + mA) for integer m . Also, 

-^</(x)<4forallx, 
4 4 


20 and specifically. 


^2^ 4 


The embedding operation =Eia„St) is implemented by the following 
algorithm: 

25 if rem(—) > — , then d = 3 — - rem(—) , 

2 4 A 


10 
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else p, = rem( — ) ; 

if (I 1> y) , then e, = sign(ei)^ ; 

if rem(—) > — , then e, = -e, : 
^A 2 

5 if a, > 0 , then =a, +e,, 

else 6, = a, -e, . 


In the above algorithm, yff is a parameter, the choice of which is dictated 
by the distortion constraint P and the energy of the channel noise Z . Also sign 
10 (x) equals +1 if the quantity 'x' is positive and sign (x) equals -1 if the quantity 'x' 
is negative. For example, 

sign(-20) = sign(-l) = -1, and sign(l 1) = sign(l) = +1. 

As an example of the embedding and detection operations, let 
15 A=[-65, -250, 19, 43, -172, 179, 178, -6], and 

S=[10, 10,-10, 10,-10, 10,-10, 10], 

A = 40,and fi = lO (Note that -^^s, for alii). 

Now B = E(A, S) is given by 

B=[-60,-255, 14, 48,-167, 180, 173,-11]. Let 
20 Z=[-4, -8, 2, -10, -5, 3, -6, -4]. Therefore, 

C=B+Z=[-64, -263, 16, 38, -172, 183, 167, -15], and 
S, = D{C) is now 

=[6,7,6,-8,2,7,-3,5]. 
Now let us consider b, - E{a, , ^, ) , for i= 1 . a, = -65, i-, = 1 0 . 

25 If re/w(^) > - , then p,- = 3 - - rew(^) , 

^A^ 2 ^' A 

refn{^) = 25 >^,p, =30-25 = 5 
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else Pi = rem(~) - — 

=10-5 = 5 

if (I ^/ 1> ^) . then = sign{e,)^ e,^^ 

if rem{^) > ^ ^, = -e,- /-e/wC^) = 25 > ~ , = -5 

5 if > 0 6; = + e^. 

else =a^~ e,. = -65 - (-5) = -60 


Now Cj = 6j + z, = -60 - 4 = -64 . For detection, 

a. = rem(—). q. = rem( ) = 24 

10 if > then 5^,. =3^-?; =24>20,5,, =30-24 = 6 

else 

4 


For Type II systems ( A = yff ) the distortion introduced, viz. B-A for 
embedding the iS in ^ (to obtain B ) will be uniformly distributed between - y 

15 and + ~ (FIGURE 3a), and the average energy of the distortion introduced in A 

will be ^ . For Type III systems, the distribution of the distortion introduced is 

depicted in FIG. 3b. The average energy of the distortion for a Type III system is 
given by 

;g'(3A-2yg) 
12A 

20 While for Type II systems s, = D(b,) , for the proposed Type III system 
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where ^. is noise due to "limiting" (limiting occurs when <Ay The distribution 

of the limiting noise is shown in FIG. 3 c, and the average energy of the limiting 

noise is given by 

12A 

5 

Optimal choice of the parameters A and for a given signal-to-noise ratio 

(snr) 

gj^^Si^ialEnergy ^ 12A 

Noise Energy Energy of Channel Noise Z 

is shown in Table 1 . In Table 1 , 

10 

SNR = lOlogjo (snr) dB, and 



From the values of A and signal energy ? , can be obtained by solving 
12A 

Table 1 - Optimal Choice of k for different SNRs 


SNR 

k 

4.77 

1.24 

3.01 

1.40 

1.76 

1.55 

0.00 

1.87 

-3.01 

2.57 

-4.77 

3.14 

-6.02 

3.59 

-6.99 

4.04 

-7.78 

4.40 

-8.45 

4.78 
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-9.03 

5.11 

-9.54 

5.41 

-10.00 

5.71 

-13.01 

8.10 

-14.77 

9.95 


The optimal parameters are chosen to minimize 

which is the normalized mean square distance between the embedded signature 
5 sequence and the detected signature sequence. The minimization performed under 
the assumption that the channel noise Z has a Gaussian distribution. If Z is zero 
mean and has a variance of cr^ , then 

1 ^^f (2i + l)A _V 

1=0 2 ^ 


10 


J 


A' 


where 


2ol 


and 

erf(0 = - fe ^ dy 


+ { 

2A 


erf 


z + 

2_ 


z ^ 


-erf 


The mapping 
15 M'.I-^S, 

in the preferred embodiment talces the following form. The bit sequence I of K bits 
is grouped into K/L L-bit symbols. Each L-bit symbol will be mapped to one of in 

AT 

2^"* basis vectors of an orthogonal transform. Thus we can embed ^^if 
NL 

symbols or bits in the sequence A. For example, 
20 ifN=8192,for 

L = 2,3, 4, 5, 6, 7, 8, 9, and 10, 

14 
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K - 8192, 6144, 4096, 2560, 1536, 896, 512, 288, and 160 bits 
respectively. 


In the preferred embodiment, L bits corresponding to each symbol are 
5 assumed to represent a decimal nimiber between 1 and 2^"' . This number is used 
as the index of the basis vector to be chosen. 

The basis vectors of a QxQ orthogonal transform where Q = 2^~^ are 
obtained from a random seed as follows. The random seed (or key) is used to 
10 generate imiformly distributed random sequence 

[0,, 0,, - ,0Q ],-^7r<0,<7C. 


The 2 random mmibers define the phase of the discrete Fourier 

transform (DFT) of a sequence H. The magnitudes of the discrete Fourier 
15 coefficients are assumed to be unity. Such a sequence H is cyclic all-pass of length 
Q. H is orthogonal to all its cyclic shifts. Such a sequence derived from a random 
seed and all its cyclic shifts form a complete basis, and can therefore be considered 
as the basis vectors of a QxQ unitary transform matrix. 


20 As an example, let Q=8. Let the - 1=3 random numbers be 

[-2.7489,-0.7854, 1.1781]. 
These random numbers describe are the angles of the Discrete Fourier Transform 
coefficients of H. The angles of the 8 coefficients of H are 

[0,-2.7489,-0.7854, 1.1781, 0, -1.1781, 0.7584, 2.7489] 
25 and their magnitudes are equal to 1. The cycUc all-pass filter H is obtained by 
inverse Discrete Fourier Transform as 

[0.2915, -0.1499, 0.3999, -0.0415, 0.5621, 0.5034, -0.2534, - 

0.3121] 


30 Each segment of length Q of the signature sequence S of length N 

carries information pertaining to one symbol between 0 and 2Q-1. 

15 
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Symbol sequence - [y^, y^, ,3^^] ; 0<x <2g-l for alii 

Q 

Signature sequence - [5p • - , ] = S 

Q 

5 The algorithm for obtaining the signature sequence is as follows 

for all i 
sign=l; 

if yi^Q 

10 sign = -l; 

else 

shifts yr, 

circularshift (sign x H, shift); 

15 For example, if 

H = [0.2915, -0.1499, 0.3999, -0.0415, 0.5621, 0.5034, -0.2534, -0.3121] 
and y^ = 2 (circular shift by 2) then, 

Sf = [-0.2534, -0.3121, 0.2915, -0.1499, 0.3999, -0.0415, 0.5621, 
0.5034]. 

20 

As an other example, if y^ =10 (circular shift by 10-8=2 followed by 
negation), then 

= [0.2534,0.3121,-0.2915,0.1499,-0.3999,0.0415,-0.5621, 
-0.5034]. 

25 

The Algorithm for the inverse mapping M"* -~> isas follows: 
Each segment of length Q of the detected sequence = [S^^ , , - * > ^ ] 

corresponds to a symbol. The embedded symbol is estimated as follows: 
HH=DFT(H) 
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for all i 
SS=DFT(5,,); 
YY=IDFT(SS.*HH); 
=index(max(abs(YY))); 

5 if(YY[>;J)<0,then 


In the above algorithm, DFT stands for Discrete Fourier Transform and 
IDFT stands for Inverse DFT. y^^ is the estimate of y^, which is the symbol 

1 0 embedded in the i'th segment of S. Finally, the binary representation of e y^^ yields 

the corresponding sequence of bits 7^. , and 



is the estimate of the hidden bit sequence I. 
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1. A method for embedding a message signal in a host signal, said method 
comprising the steps of: 

5 (a) embedding said message signal into said host signal, thereby producing 

a stego signal; and 

(b) detecting an estimate of said message signal from said stego signal; 
provided that said detecting step (b) is not an exact inverse of said 
embedding step (a), and said host signal cannot be exactly extracted 
10 from said stego signal. 

2. The method according to claim \, wherein said stego signal is corrupted or 
distorted prior to detecting step (b). 

15 3. The method according to claim 1, wherein said embedding step (a) produces a 
value bi in said stego signal from a value in said host signal, and wherein 

said embedding step (a) comprises limiting to a limit value y , a magnitude of 

difference between and a, . 


20 4. The method according to claim 1, wherein said embedding step (a) employs a 
continuous periodic function to produce said stego signal, and wherein said 
detecting step (b) employs said continuous periodic function to produce said 
estimated message signal. 

25 5. The method according to claim 4, wherein said continuous periodic function is 
a triangular function. 


6. The method according to claim 5, wherein said triangular function f(x) has a 
period A , and 
30 wherein: 

-~^/(x)<4forallx; 
4 4 
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/(0) = -^; and 

^^2^ 4 


7. The method according to claim 1, wherein said embedding step (a) produces a 
value in the said stego signal from a value in said host signal and a value 

5 5( in said message signal, such that said embedding step (a): 

(i) is subject to a maximum distortion constraint P, 

(ii) employs a continuous periodic function having period A , and 

(iii) is represented by the function = £(0^ .s^), and employs an algorithm 
as follows: 

1 0 if rem{—) > — , then = 3 ~ - rem{~) , 

^A^ 2 ^' 4 'a^ 

else P; = rem(—) ; 

^' A 4 

if (I \> ^) , then = sign(e,)^; 

q,^rem{rj-)\ 
A 

15 if > y , then - -e^ ; 

if > 0 , then b^ = ; 
else b^=a^~-ef. 


8. The method according to claim 2, wherein a value b^ in said stego signal is 
20 modified after said embedding step (a) to yield a value in said corrupted or 

distorted stego signal, and wherein said detecting step (b): 

(i) produces a value in said estimated message signal from a value c, in 
said distorted stego signal, 

(ii) employs a continuous periodic finiction having a period A , and 
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(iii) is represented by the function s^^ = D{Ci) , and employs an algorithm 

as follows: 

A 

if ?i>^.then;y,, =3^-^^, 
5 else 

4 

9. The method according to claim 2, wherein 

said host signal is a sequence , for z = 1 to N ; 

said message signal is a sequence , for z = 1 to N; 
1 0 said stego signal is a sequence , for z = 1 to N; 

said corrupted or distorted stego signal is a sequence , for z = 1 to N; 
and 

said estimated message signal is a sequence s^^, for z = 1 to A'' 


15 10. The method according to claim 1, wherein said embedding step (a): 

(i) imposes a limit ~^ magnitude difference between a value in 

said stego signal that is produced from a value in said host 
signal; 

(ii) employs a continuous periodic function having a period A to 

20 produce said stego signal, wherein such said limit and said period A are 

chosen to minimize a mean square distance between said message signal 
and said estimated message signal, subject to a maximum distortion 
constraint P of said embedding step (a). 

25 11. A method for mapping K information bits to a message signal s^, 
z = 1 to N , said method comprising the steps of: 


20 


wo 02/13436 PCT/USOl/24468 
grouping said K information bits together to represent one of 2^ symbols, 
wherein each said 2^ symbol is mapped to a basis vector or its negative of a 
2^"* X 2^"* orthogonal transform matrix. 

5 12. The method according to claim 11, wherein said orthogonal transform matrix 
is obtained from a cyclic all-pass filter and its ciruclar shifts. 


13. The method according to claim 12, wherein said cyclic all-pass filter is 
obtained from a key, 

10 
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FIGURE 3 
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